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ABSTRACT 


Schaffner's logic of comparative theory evaluation is 
criticised for an inappropriate analysis of ad hocness. 
An alternative analysis, based on Zahar's account of 
novelty, is given and extended to the case of multiple 
Successful predGictions by a theory. The application of 
the method to the appraisal of quantitative prediction 
is discussed, 


— 


The Logic of Comparative Theory Evaluation * 

pecs 

Li The Bayesian Analysis of Ad hocness. 
ee is of Ad hocness 


In a recent note Schaffner (C1974) ) has given a formal discussion 
of the notion of ad hocness in terms of a Bayesian model for the 
appraisal of theories. Schaffner develops his general ideas in 
the context of a critique of Zahar's {1973| which was concerned 
with the particular problem of comparing the Einstein and Lorentz 
research programmes. Zahar sugeests the following analysis of 


ad hocness”: 


_Ad hoc,: A theory is said to be ad hoc, if it has no 


novel consequences as compared with its predecessor. 


Ad hoc,: fa theory}. is ad hoc, if none of its novel 
predictions have been actually 'verified', 


Ad hoc,: (Pies? theory is said to be ad hoc, if it is 
obtained from its predecessor through a modification of 
the auxiliary hypotheses which does not accord with the 
spirit of the heuristic of the programme, 


Zahar explains the meaning of novelty as follows 
A fact will be considered novel with respect to a given 
hypothesis if it did not belong to the problem-situation 
which governed the construction of the hypothesis. 


Schaffner begins his elucidation by discussing the notions of 
ad hoc, and ad hoc... The first he describes as a logical dream, 
Since the novel consequences of a theory cannot in practice be 
"surveyed", so the’ question of whether a theory is ad hoc, can 
only be discussed relative to the extent to which novel 
consequences have been looked for at the particular epoch of 


the evaluation. Ad hoc, Schaffner claims to be "vague to the 
is "close to 


point of inapplicability". For Schaffner ad hoc, 
a ee ae 
* IT am grateful to Donald Gillies, Jon Dorling and Noretta Koertge 


for helpful comments ana suggestions. The present work formed part 
of a paper read in Professor Watkins’ seminar at The London School 


of Economics in January 1976. 


the sense in which ad hoc is used in science" and he announces 
that his Bayesian analysis will be brought to bear on this 
sense of ad hoc, 


Denoting by P(T/bX e) the probability of a theory T to be true 
in the light of background knowledre b and the positive outcome 
e of some experiment: not part of b, by p(T/b) the prior 
assessment of T, by p(e/TXb) the probability of obtaining e 
given T and b, and by p(e/b) the probability of obtaining e on 
the basis of background knowledge alone, Schaffner writes 


p(T/b&e) = BOE Ble (Te) (2) 
ple/b) 


If T explains 6 we can set pl(e/TX) = 1 so in this case we 
obtain 

p(T/EZe) = p(t/e) /ple/t) (2) 
Schaffner proceeds to discuss ad hocness as a property of an 
hypothesis as/constituent part of a theory, but in order 
to keep the argument as simple as possible we shall follow 
Zahar in considering the ad hoecness of theories. Translated 
into these terms Schaffner's idea is that a theory T gives 
an ad hoc explanation of an experimental result e if p(T/é ) 
is close to zero and ple/2) is significantly larger than 
p(T/2 ). The argument for p(T/2 ) being small is that it 
has no "theoretical support" (or indeed empirical support 
other than e itself), This looks suspiciously like a 
reference to ad hoc,, and Schaffner now appears to be claiming 
that a theory is ad hoc, partly in virtue of its being ad hoc. 
So he is not really giving an independent analysis of ad hoc, 
at all. 


The lack of novelty in the prediction of e is associated by 
Schaffner with a high value for ple/2-),. On Zahar's account 

this is a necessary condition for lack of novelty, but not a 
sufficient condition. We proceed to show how using Zahar's notion 


“ 
of novelty cna can get an "internal Bayesian analysis of ad hoc.,. 


We express Bayes'ts theorem in the following familiar way 


p(T/b& e) = p(T/b). p(e/TXb) C3) 


p(e/TR b)p(T/b) Soa POWs 


where ~T denotes the negation of T. 


We follow Schaffner in interpreting p(A/B) as the degree of 
belief that A is true given that B is true. Note also that e 
is supposed in equation (3) to refer to a prediction made by 
the theory T (we could perhaps write en to emphasize this 
important point) so p(e/~T&b) means the probability that the 
prediction En derived from the theory T is a true prediction 
given that the background information b is true but the 

theory T is false (i.e., its consequences are not guaranteed to 
be true although "by accident" they may be true). 


Writing p(T/b) = x, p(eAT& db) 


and taking p(e/T£ b) = 1 and using p(eT/b) = 1-x 
x 
x +¢(1- x) 


p(T/bxe) = (4) 


We define an enhancement ratio Y by 


(T/b¥ e) 
* aes OF (55 


whence using (4) we obtain the simple result 


7 1 
1 semper (5) 


We can now explain that if a theory T is ad hoc, with respect to 
the experiment e then € = 1, i.e. the explanation of e by T in 

no way depends on the truth or falsehood of T, both of which 
eventualities lead with certainty to the result e. This is just 
what a scientist means when he says T was an ad hoc explanation of 
€, namely T was devised for the express purpose of explaining e, 


so the explanation of e is puaranteed independent of whether 

T is true or false. To show the consistency of our analysis 

if we put £ = 1 in (5) we pet Y = 1, so the posterior and prior 
probabilities of T are equal (there is no enhancement) and this 
again is just what we expect from an ad hoc explanation of e, 
namely e itself gives us no additional information for assessing 
the truth of ..° , 

Notice that p(e/b) = x +£&(1 - x) is equal to unity ir § = 

but that p(e/2.) = 1 is achieved for x = 1 whatever the rvalue 
of © , so the iecaear ana of novelty in terms of low ple/é) 
(Schaffner) and small £ (Zahar) are by no means equivalent. 


On our analysis if x2€¢1, then Y = Y€ , so in this case we 

met a big enhancement and the theory is far from being ad hoc. 
Noting that under these conditions ple/e )x &, we see that this is 
a situation in which Schaffner would claim that the theory was 

ad hoc, which highlirhts the way in which his analysis differs 

from ours. Effectively Schaffner requires Y X to be small as 

his condition of ad hocness. He is thus concerned with the 
absolute value of the probability of a theory after it has 
explained some experimental finding. If this absolute value 

is still small the theory is to be regarded as an ad hoc capone bien 
of the experiment. On our account the important aspect in 
assessing ad hocness for this case is not the absolute value of 

the probability but the enhancement ratio. Our point against 
Schaffner is not that his analysis may not explicate some 
legitimate sense of the ambiguous appellation ad hoc, but that it 
fails to explicate Zahar's very important notion of ad hocy. There 
is no inconsistency here on Schaffner's part, since he finds 
Zahar's account of ad hoc, inadequate in respect of the definition 
of novelty involved with aes See historical associations, 
but we would maintain that Zahar's sense of ad hoc, is the one that 
ought to be explicated since it is the one most importantly used 


in science. 


Pam The Case of Multiple Predictions 


To develop our analysis a little further we can consider how a 
theory builds up a favourable appraisal as it makes a number of 
Successful predictions Cys Cneeee, Say. Denote by Pe the 
posterior probability p(T/b¥e, ¥en... Re.) after s successful 
predictions, Assuming the predictions are quite independent 
and for simplicity are ali associated with the same value of & , 


we can clearly write 


Po = y™) , LYS yO), n 


fo) 
where Po = X according to our previous notation 
|: 
and a (s) pe, Coe 
Psy t-g)+eE 
_ (s+1) 
Poe 5 ¥ S 


The solution of this recursion is by inspection or more simply 
Z Pe 
by replacing ¢ by & in the formula for Py (see (4) above), We 


obtain 


a ae 
120, ae (6) 
We can also ask what is the probability for the (n+1)% 
prediction Ce being correct if the theory has already made n 


Successful predictions Cyr, Denoting this Probability 
by o(@ n+1) we clearly obtain the result 
1- € 


p({ne a) = © x ——, —_____ 
1-7 4+ EM (7) 

If £ is a small quantity (i.e. <1) which will be the case if T 

is non-ad hoc, with respect to all the predictions, we can 

write the following formulae which will be perfectly satisfactory 

for the subsequent discussion 


eee (8) 


p(nti)~ € + Pn ‘ (9) 


CW) sy de (10) 
v . ine Ph cee i 


We note the following features 
(1) The value .of pn depends entirely on the ratio EM x 
For €"/x >> 1, pm Kl 
and for £"/x 1, Pr 1 
At the critical point ¢™ = xX, we have pn & 1/2, 


So if initially x<€ as n increases Pn will rise 
steeply as we reach the value n = Lax/bn € : 


(2) So long as pn&£, we have p(€n+1) <€, but as pn builds 
up towards unity, so does p(@n+41), 


(3) So long as Pn -4 €E 5 a Ve » but as bn - 1 builds 


up towards unity the enhancement factor Y (n) also 
tends to unity. 


To take a concrete case we illustrate in the accompanying figure 
pn and p(£n+1) as functions of n for the particular choice 
x=2=0.01, € =oO.1, 
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of n this condition will fail. For example in the case of the 

anomalous magnetic moment of the electron the effect of hadronic 
couplings introduce theoretical uncertainties which would 
ultimately make the prediction of the theory unreliable.” 


Me Conclusion 


The concept of a novel prediction plays a very important part 

in the way scientists assess their confidence in theories. Of 
course on a purely instrumentalist view of theories the question 
of novelty in Zahar's sense is of no consequence. But to a 
realist it is only in virtue of novel successful predictions that 
commitment to a theory can find rational justification. Our 
analysis has shown how the logic of comparative theory evaluation 
can be viewed if degrees of commitment are governed by Bayesian 


rationality constraints. 


FOOTNOTES 


1. 


zahar fo73] p.101. In his fas7a] Zahar rephrases 

his definition in terms of a notion of empirical 

non~ad hocness expressed as a three-place relation 
between an observation statement, a theory, and a 
heuristic. See also in this connection the very clear 
account of empirical support given by Worrall in 

his fo75] - However this more recent work has somewhat 
obscured the important distinctions drawn by Zahar in 


nis [i973]. 
Zahar [2973] p.103. 


What we have shown is that & = 1 is a necessary condition 
for T to be an ad hoc, explanation of e. To justify 

& = 1 as a sufficient condition we must invoke a principle 
of insufficient reason, viz.,if there is no reason for 

the community of scientists to entertain with non-vanishing 
prior probability only theories which explain e, then they 
will not so constrain their choice of alternative theories. 


The successful detailed quantitative predictions of a 
theory in respect of phenomena quite different from those 
which the theory was originally proposed to deal with haye 
always attracted the attention of scientists. To take an 
example at random, in referring to his early work on the 
ground-state of Helium Hylleraas comments in his [2963 ] 
(p.42) "The end result of my calculation was...greatly 
admired and thought of as almost a proof of the validity 
of wave mechanics...in the strict numerical sense". 


Van Dyck et al. (fr977]). 


See value quoted in Calmet et al. ((2977|). For a good 
account of the fluctuating agreement between theory and 
experiment the reviews by Lautrup, Peterman and de Raphael 
( (19721) or Rich and Wesley ( fr972\) may be consulted. 


We may refer to Shimony's concept of commitment to a 
theory (see nis [1970] pp.94-95). The degree of 
commitment measures the scientist's belief that the 
theory T belongs to the equivalence class of all theories 
which give the same true observational predictions within 
its domain of current experimentation and that the "true" 
theory "generalizes" in some sense the concepts embodied 
in T. Commitment measures our belief, not that a theory 
is true, but that it points the way to the truth. 
Naturally this second aspect of commitment cannot be 
assessed via novel predictions within the current 
experimental domain. 


It is easy to see that our confidence in a theory at 

a given level of accuracy for agreement between 

theory and experiment does not depend on the particular 
scale of notation used to express the result. 


According to Rich and Wesley ( hoz2}) the known hadronic 
contribution to the electron anomaly would affect the 
tenth significant figure. 
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